Recent progress in hardware and methodology for training neural networks has ushered in a new generation of large networks trained on abundant data. These models have obtained notable gains in accuracy across many NLP tasks. However, these accuracy improvements depend on the availability of exceptionally large computational resources that necessitate similarly substantial energy consumption. As a result these models are costly to train and develop, both financially, due to the cost of hardware and electricity or cloud compute time, and environmentally, due to the carbon footprint required to fuel modern tensor processing hardware. In this paper we bring this issue to the attention of NLP researchers by quantifying the approximate financial and environmental costs of training a variety of recently successful neural network models for NLP. Based on these findings, we propose actionable recommendations to reduce costs and improve equity in NLP research and practice.

The rapid growth of machine translation (MT) systems has necessitated comprehensive studies to meta-evaluate evaluation metrics being used, which enables a better selection of metrics that best reflect MT quality. Unfortunately, most of the research focuses on high-resource languages, mainly English, the observations for which may not always apply to other languages. Indian languages, having over a billion speakers, are linguistically different from English, and to date, there has not been a systematic study of evaluating MT systems from English into Indian languages. In this paper, we fill this gap by creating an MQM dataset consisting of 7000 fine-grained annotations, spanning 5 Indian languages and 7 MT systems, and use it to establish correlations between annotator scores and scores obtained using existing automatic metrics. Our results show that pre-trained metrics, such as COMET, have the highest correlations with annotator scores. Additionally, we find that the metrics do not adequately capture fluency-based errors in Indian languages, and there is a need to develop metrics focused on Indian languages. We hope that our dataset and analysis will help promote further research in this area.

Finetuning image-text models such as CLIP achieves state-of-the-art accuracies on a variety of benchmarks. However, recent works like WiseFT (Wortsman et al., 2021) and LP-FT (Kumar et al., 2022) have shown that even subtle differences in the finetuning process can lead to surprisingly large differences in the final performance, both for in-distribution (ID) and out-of-distribution (OOD) data. In this work, we show that a natural and simple approach of mimicking contrastive pretraining consistently outperforms alternative finetuning approaches. Specifically, we cast downstream class labels as text prompts and continue optimizing the contrastive loss between image embeddings and class-descriptive prompt embeddings (contrastive finetuning). Our method consistently outperforms baselines across 7 distribution shifts, 6 transfer learning, and 3 few-shot learning benchmarks. On WILDS-iWILDCam, our proposed approach FLYP outperforms the top of the leaderboard by $2.3\%$ ID and $2.7\%$ OOD, giving the highest reported accuracy. Averaged across 7 OOD datasets (2 WILDS and 5 ImageNet associated shifts), FLYP gives gains of $4.2\%$ OOD over standard finetuning and outperforms the current state of the art (LP-FT) by more than $1\%$ both ID and OOD. Similarly, on 3 few-shot learning benchmarks, our approach gives gains up to $4.6\%$ over standard finetuning and $4.4\%$ over the state of the art. In total, these benchmarks establish contrastive finetuning as a simple, intuitive, and state-of-the-art approach for supervised finetuning of image-text models like CLIP. Code is available at https://github.com/locuslab/FLYP.

The primary obstacle to developing technologies for low-resource languages is the lack of representative, usable data. In this paper, we report the deployment of technology-driven data collection methods for creating a corpus of more than 60,000 translations from Hindi to Gondi, a low-resource vulnerable language spoken by around 2.3 million tribal people in south and central India. During this process, we help expand information access in Gondi across 2 different dimensions (a) The creation of linguistic resources that can be used by the community, such as a dictionary, children's stories, Gondi translations from multiple sources and an Interactive Voice Response (IVR) based mass awareness platform; (b) Enabling its use in the digital domain by developing a Hindi-Gondi machine translation model, which is compressed by nearly 4 times to enable it's edge deployment on low-resource edge devices and in areas of little to no internet connectivity. We also present preliminary evaluations of utilizing the developed machine translation model to provide assistance to volunteers who are involved in collecting more data for the target language. Through these interventions, we not only created a refined and evaluated corpus of 26,240 Hindi-Gondi translations that was used for building the translation model but also engaged nearly 850 community members who can help take Gondi onto the internet.

Reinforcement Learning (RL) algorithms are known to scale poorly to environments with many available actions, requiring numerous samples to learn an optimal policy. The traditional approach of considering the same fixed action space in every possible state implies that the agent must understand, while also learning to maximize its reward, to ignore irrelevant actions such as $\textit{inapplicable actions}$ (i.e. actions that have no effect on the environment when performed in a given state). Knowing this information can help reduce the sample complexity of RL algorithms by masking the inapplicable actions from the policy distribution to only explore actions relevant to finding an optimal policy. This is typically done in an ad-hoc manner with hand-crafted domain logic added to the RL algorithm. In this paper, we propose a more systematic approach to introduce this knowledge into the algorithm. We (i) standardize the way knowledge can be manually specified to the agent; and (ii) present a new framework to autonomously learn these state-dependent action constraints jointly with the policy. We show experimentally that learning inapplicable actions greatly improves the sample efficiency of the algorithm by providing a reliable signal to mask out irrelevant actions. Moreover, we demonstrate that thanks to the transferability of the knowledge acquired, it can be reused in other tasks to make the learning process more efficient.

Explainability has been widely stated as a cornerstone of the responsible and trustworthy use of machine learning models. With the ubiquitous use of Deep Neural Network (DNN) models expanding to risk-sensitive and safety-critical domains, many methods have been proposed to explain the decisions of these models. Recent years have also seen concerted efforts that have shown how such explanations can be distorted (attacked) by minor input perturbations. While there have been many surveys that review explainability methods themselves, there has been no effort hitherto to assimilate the different methods and metrics proposed to study the robustness of explanations of DNN models. In this work, we present a comprehensive survey of methods that study, understand, attack, and defend explanations of DNN models. We also present a detailed review of different metrics used to evaluate explanation methods, as well as describe attributional attack and defense methods. We conclude with lessons and take-aways for the community towards ensuring robust explanations of DNN model predictions.

Deep Ensemble Convolutional Neural Networks has become a methodology of choice for analyzing medical images with a diagnostic performance comparable to a physician, including the diagnosis of Diabetic Retinopathy. However, commonly used techniques are deterministic and are therefore unable to provide any estimate of predictive uncertainty. Quantifying model uncertainty is crucial for reducing the risk of misdiagnosis. A reliable architecture should be well-calibrated to avoid over-confident predictions. To address this, we propose a UATTA-ENS: Uncertainty-Aware Test-Time Augmented Ensemble Technique for 5 Class PIRC Diabetic Retinopathy Classification to produce reliable and well-calibrated predictions.

3D牙齿分割是数字正畸技术的重要任务。已经提出了几种深度学习方法，用于从3D牙科模型或口腔内扫描中进行自动牙齿分割。这些方法需要注释的3D口内扫描。手动注释3D口腔内扫描是一项费力的任务。一种方法是设计自学方法来减少手动标签工作。与其他类型的点云数据（例如场景点云或形状点云数据）相比，3D牙齿点云数据具有非常规定的结构和强大的形状。我们查看可以从单个3D口内扫描中学到多少代表性信息。我们借助十种不同的方法来定量评估，其中六种是通用点云分割方法，而其他四种是特定于牙齿分割的方法。令人惊讶的是，我们发现，在单个3D口内扫描训练中，骰子得分可以高达0.86，而完整的训练组可得分为0.94。我们得出的结论是，分割方法可以从单个3D牙齿点云扫描中学习大量信息，例如数据增强。我们是第一个从单个3D口内扫描中进行定量评估并证明深度学习方法的表示能力的人。这可以通过最大程度地利用可用的数据来实现在极端数据限制方案下构建牙齿分割的自学方法。

姿势图优化是同时定位和映射问题的一种特殊情况，其中唯一要估计的变量是姿势变量，而唯一的测量值是施加间约束。绝大多数PGO技术都是基于顶点的（变量是机器人姿势），但是最近的工作以相对方式参数化了姿势图优化问题（变量是姿势之间的变换），利用最小循环基础来最大程度地提高范围的稀疏性。问题。我们以增量方式探索周期基础的构建，同时最大程度地提高稀疏性。我们验证一种算法，该算法逐渐构建稀疏循环基础，并将其性能与最小循环基础进行比较。此外，我们提出了一种算法，以近似两个图表的最小周期基础，这些图在多代理方案中常见。最后，姿势图优化的相对参数化仅限于使用SE（2）或SE（3）上的刚体变换作为姿势之间的约束。我们引入了一种方法，以允许在相对姿势图优化问题中使用低度测量值。我们对标准基准，模拟数据集和自定义硬件的算法进行了广泛的验证。

我们研究了改进的多臂匪徒（IMAB）问题，其中从手臂获得的奖励随着收到的拉力数量而增加。该模型为教育和就业等领域中的许多现实世界问题提供了优雅的抽象，在这种领域中，关于机会分配的决定可能会影响社区的未来能力以及它们之间的差异。在这种情况下，决策者必须考虑她的决策对未来奖励的影响，除了随时最大化其累积奖励的标准目标。在许多这些应用中，决策者的时间范围未知，这激发了在技术上更具挑战性的地平线环境中对IMAB问题的研究。我们研究了地平线 - 统一环境中两个看似相互冲突的目标之间产生的紧张：a）根据武器的当前奖励，在任何时候最大化累积奖励，b）确保具有更好的长期奖励的武器获得足够的机会即使他们最初的奖励很低。我们表明，令人惊讶的是，在这种情况下，这两个目标是相互对齐的。我们的主要贡献是对IMAB问题的任何时间算法，它可以获得最佳的累积奖励，同时确保武器在足够的时间内发挥其真正的潜力。由于缺乏机会，我们的算法减轻了最初的差异，并继续拉动手臂直到停止改善。我们通过证明a）imab问题的任何算法来证明我们的算法的最佳性，无论其功利主义，无论多么有效，都必须遭受$ \ omega（t）$政策后悔和$ \ omega（k）$竞争比率相对于最佳的比例离线政策和b）我们算法的竞争比率为$ O（k）$。